Dextrous Haptic Interface for Jack

This paper presents a simulation system which integrates a realtime hand sensing and force-feedback interface (the Rutgers Master II) with a full body modeling software (JACK ). The RM– II system and the Polhemus Fastrak 3D magnetic tracker were initially integrated with “JACK" full-body simulation software. The system performance (graphics update rates, sensor reading updates) is evaluated on two graphic workstations (SGI High Impact and SGI Infinite Reality). Next we propose the concept of a Virtual Human Agent enhanced with haptic feedback. This concept is illustrated by a simulation developed using the RM–II system, JACK software library and a speech recognition engine. INTRODUCTION Many haptic systems are currently limited to providing feedback to only one location of user’s body (mainly the hand) [1]. Due to advances in VR technology [2] and increased workstation computation power the idea of a full body haptic suit comes closer to reality. Virtual humans can be used as graphic interfaces for such complex VR simulations. They have been used mostly for human factors design, ergonomics visualization and virtual prototyping and less in real-time simulation systems. Therefore some limitations are inherent to the currently available Virtual Humans when used in real-time applications. Designing complex systems with full-body tracking and force feedback require very powerful simulation tools. However, tracking and feeding back forces for those parts of the body with the highest haptic activity for a given application can be currently realized with a single workstation and additional VR I/O devices. This paper presents an implementation of a Virtual Human enhanced with force feedback for the hand. The following section presents the integration of the JACK software with the RM–II haptic interface. The system performance is evaluated for two platforms, medium and high-end graphic workstations. A proof-of-concept demonstration of a Virtual Human Agent (VHA) performing tasks in a machine shop is described next. Multimodal input needed to communicate with the VHA is outlined briefly. Graphics and sensors update rates are evaluated subsequently. Concluding remarks are given in the last section. THE HAPTIC INTERFACE SYSTEM